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I. INTRODUCTION 



The Standard Model (SM), based on the local gauge symmetry SU (3) c x SU{2) L xU (l) y , 
is very successful in describing all the experimental results below the TeV scale. It is an 
excellent effective field theory, but it is widely believed not to be the final theory. Discovery 
of new particles is highly anticipated at the Large Hadron Collider (LHC). The most likely 
and reasonably well motivated candidates are supersymmetric particles, and extra Z' boson. 
However, it is important to explore other alternatives or entirely new possibilities at the 
current and future LHC. 

In the SM, we have fermions (spin 1/2) and scalars (Higgs fields) (spin 0) which do not 
belong to adjoint representations under the SM gauge symmetry. Can we also have TeV 
scale gauge bosons (spin 1) belonging to the non-adjoint representations under the SM gauge 
symmetry? Can we achieve the (partial) grand unified theory at the TeV scale? Can we 
construct a renormalizable theory realizing such a possibility which can be tested at the 
LHC? These are very interesting theoretical questions that we shall address in this work. 
Discovery of such gauge bosons around the TeV scale at the LHC will open up a new window 
for our understanding of the fundamental theory describing the nature. 

How can we construct a consistent theory involving the massive vector bosons which do 
not belong to the adjoint representations under the SM gauge symmetry? If the massive 
vector bosons are not the gauge bosons of a symmetry group, there are some theoreti- 
cal problems from the consistency of quantum field theory, for instance, the unitarity and 
renormalizability When the gauge symmetry is spontaneously broken via the Higgs 
mechanism, the interactions of the massive gauge bosons satisfy both the unitarity and the 
renormalizability of the theory {2, 3|. Thus, the massive vector bosons must be the gauge 
bosons arising from spontaneous gauge symmetry breaking. 

As we know, a lot of models with extra TeV scale gauge bosons have been proposed 
previously in the literature. However, those massive gauge bosons either belong to the 
adjoint representations or are singlets under the SM gauge symmetry j^-ll]. For example, 
in the top color model |4-|6|, the colorons belong to the adjoint representation of the SU(3)c', 
in the top flavor model (7|,|8j, the extra W and Z' bosons belong to the adjoint representation 
of the SU(2)l, while in the U(l)' model [9] or top hypercharge model 10|, the new Z' boson 
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is a singlet under the SM gauge symmetry. In the Grand Unified Theories such as 577(5) 



and SO (10) 12|, [l3[], there are such kind of massive gauge bosons. However, their masses 
have to be around the unification scale ~ 10 16 GeV to satisfy the proton decay constraints. 

Some years ago, two of us (TL and SN) had proposed a class of models where the gauge 
symmetry is Q = U i G t x SU(3)' C x SU(2)' L x U(1)' Y Q. The quantum numbers of the 
SM fermions and Higgs fields under the SU(3)' C x SU(2)' L x U(1)' Y gauge symmetry are 
the same as they have under the SM gauge symmetry SU(3)c x SU(2) L x U(1)y, while 
they are all singlets under Hi^i- Hence rL^« * s the hidden gauge symmetry. After the 
gauge symmetry Q is spontaneously broken down to the SM gauge symmetry at the TeV 
scale via Higgs mechanism, some of the massive gauge bosons from the Q breaking do 
not belong to the adjoint representations under the SM gauge symmetry. In particular, 
a concrete SU(5) x SU(3)' C x SU(2)' L x U(1)' Y has been studied in detail. However, the 
corresponding (X^, Y^) massive gauge bosons are meta-stable and behave like the stable 
heavy quarks and anti-quarks at the LHC [14]. Thus, an interesting question is whether we 
can construct the SU(5) x SU (3)' c xSU (2)' L x U(1)' Y models where the (X^, Y^) gauge bosons 
can decay and produce interesting signals at the LHC. By the way, the six-dimensional 
orbifold non-supersymmetric and supersymmetric 577(5) and 577(6) models with low energy 



gauge unification have been constructed previously [15j-[l7j. However, there is no direct 
interactions between the (X^, Y^) particles and the SM fermions. 

As we pointed out above, the top color model ^-6|, top flavor model (3, lsj|, and top 
hypercharge model [10] have been constructed before. Because of the proton decay problem 
and quark CKM mixings, etc, the real challenging question is whether we can construct 
the top 577 (5) model as the unification of these models. Consequently, we can explain the 
charge quantization for the third family, and probe the baryon and lepton number violating 
interactions involving the third family at the LHC. Such a model was proposed by us recently 



18] , and its implications for LHC was briefly explored. 

In this paper, we shall propose two such models: the minimal and the renormalizable top 
577(5) model where the 577(5) x SU(3)' C x SU(2)' L x U(1)' Y gauge symmetry is broken down 
to the SM gauge symmetry via the bifundamental Higgs fields at low energy. The first two 
families of the SM fermions are charged under SU(3)' C x SU(2)' L x U(1)' Y while the third 
family is charged under 577(5). In the minimal top SU(5) model, we show that the quark 
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CKM mixing matrix can be generated via dimension-five operators, and the proton decay 
problem can be solved by fine-tuning the coefficients of the higher dimensional operators 
at the order of 10~ 4 . In the renormalizable top SU(5) model, we can explain the quark 
CKM mixing matrix by introducing vector-like particles, and we do not have proton decay 
problem. In these models, the non-unification of the three SM couplings are remedied, 
because three SM couplings g 3 , g 2 , g\ are now combinations of (#5, #3), (#5, #2)) (#5) 9\)i anc ^ 
need not be unified. Since the models have baryon and lepton number violating interactions, 
it might be useful in generating the baryon asymmetry of the Universe. In our models, since 
the third family quark lepton unification is at the TeV scale, we can probe the new (X M , F M ) 
gauge bosons at the LHC through their decays to the third family of the SM fermions. 

Our paper is organized as follows. In section II, we discuss the two models and their 
formalism. In section III, we discuss in detail the phenomenological implications of the 
models. These include the productions and decays of the X and Y gauge bosons at the LHC 
energies of 7, 8 and 14 TeV, their decay modes, and the signals for the final states. We 
also discuss the LHC reach for the masses of these particle for various LHC energies and 
luminosities. Section IV contains our summary and conclusions. 

II. THE MINIMAL AND RENORMALIZABLE TOP SU(S) MODELS 

We propose two non-supersymmetric top 577(5) models where the gauge symmetry is 
S77(5) x SU(3)' C x SU(2)' L x U(1)' Y . The first two families of the SM fermions are charged 
under SU(3)' C x SU(2)' L x U(1)' Y while the third family is charged under S77(5). We denote 
the gauge fields for S77(5) and SU(3)' C x SU(2)' L x U(1) Y as and A^ respectively, 
and the gauge couplings for 577(5), SU(3)' C , SU(2)' L and U(1)' Y are g 5 , g' 3 , g' 2 and g' Y , 
respectively. The Lie algebra indices for the generators of S77(3), SU(2) and U(l) are 
denoted by a3, a2 and al, respectively, and the Lie algebra indices for the generators of 
SU(5)/(SU(3) x 51/(2) x 17(1)) are denoted by a. After the SU(5) x SU(3)' C x SU{2)' L x U(1)' Y 
gauge symmetry is broken down to the SM gauge symmetry SU(3)c x SU(2)l x U(1)y, we 
denote the massless gauge fields for the SM gauge symmetry as A™, and the massive gauge 
fields as 77™ and A a . The gauge couplings for the SM gauge symmetry SU(3) C) SU(2) L and 
U(1)y are g 3 , g 2 and gy, respectively. 
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To break the SU(5) x SU(3)' C x SU(2)' L x U(1)' Y gauge symmetry down to the SM 



gauge symmetry, we introduce two bifundamental Higgs fields Ut and Ud [14| . Let us 
explain our convention. We denote the first two family quark doublets, right-handed up- 
type quarks, right-handed down-type quarks, lepton doublets, right-handed neutrinos, right- 
handed charged leptons, and the corresponding Higgs field respectively as Qi, Uf, D°, L iy 
iVf, Ef, and H, as in the supersymmetric SM convention. We denote the third family SM 
fermions as F 3 , f 3 , and N£. To give the masses to the third family of the SM fermions, we 
introduce a S77(5) anti-fundamental Higgs field $ = (H T ,H'). We also need to introduce 
a scalar field XT if we require that the triplet Higgs H' T have mass around the S77(5) x 
SU(3)' C x SU(2)' L x U(1)' Y gauge symmetry breaking scale. However, it is not necessary, and 
we will explain it in the following. In addition, note that the neutrino PMNS mixings can be 
generated via the right-handed neutrino Majorana mass mixings, we propose two top S77(5) 
models which can generate the mass for the possible pseudo-Nambu-Goldston boson (PNGB) 
during the gauge symmetry breaking and generate the quark CKM mixings. In the minimal 
top SU(5) model, we consider the dimension-five non-renormalizable operators and fine- 
tune some coefficients of the higher dimensional operators at the order 10~ 4 to suppress the 
proton decay. In the renormalizable top 577(5) model, we introduce the additional vector- 
like particles. To give the PNGB mass, we introduce a scalar field XU in the S77(5) anti- 
symmetric representation. And to generate the quark CKM mixings while not to introduce 
the proton decay problem, we introduce the vector-like fermionic particles (Xf, Xf c ) and 
(XD, XD C ). Note that the SU(3)' C x SU(2)' L x U{l)' Y gauge symmetry can be formally 
embedded into a global S77(5)' symmetry, and to do that, we introduce the vector-like 
particles (XL, XL C ) as well. The complete particle content and the particle quantum 
numbers under S77(5) x SU(3)' C x SU{2)' L x U(1)' Y gauge symmetry are given in Table HI 

To give the vacuum expectation values (VEVs) to the bifundamental Higgs fields Ut and 
Ud, we consider the following Higgs potential 



V = -m z T \U T \ - m D \U D \ + X T \U T \ Z + X D \U D \ Z + X TD \U T \\U D \ 

Vtd 



+ 



A T $U T XT^ + A D <$>U D H ] + ^U T U 2 D + H.C. 



(2.1) 



where M* is a normalization mass scale. 

A few remarks are in order. First, with XT particle, the Higgs triplet H' T will have mass 
around the SU(5) x SU(3)' C x SU(2)' L x U(1)' Y gauge symmetry breaking scale, as given by 
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TABLE I: The complete particle content and the particle quantum numbers under SU(5) x 
SU(3)' C x SU(2)' L x U(1)' Y gauge symmetry in the top SU(5) model. Here, i = 1, 2, and k = 1, 2, 3. 



Particles 


Quantum Numbers 


Particles 


Quantum Numbers 




(1; 3, 2, 1/6) 


u 


(1; 1,2,-1/2) 




(1; 3, 1,-2/3) 




(1; 1,1,0) 


A 5 


(1; 3, 1,1/3) 




(i; 1,1,1) 


^3 


(10; 1,1,0) 


7 3 


(5; 1,1,0) 


H 


(1; 1,2,-1/2) 


$ 


(5; 1,1,0) 


U T 


(5; 3, 1,1/3) 


u D 


(5; 1,2,-1/2) 


XT 


(1; 3, 1,1/3) 


xu 


(10; 1,1,-1) 


Xf 


(5; 1,1,0) 


Xf 


(5; 1,1,0) 


XD 


(1; 3, 1,-1/3) 


XD 


(l; 3, 1,1/3) 


XL 


(1; 1,2,-1/2) 


~XL 


(i; 1,2, 1/2) 



the above A? term. However, it is still fine even if we do not introduce the XT field. Let 
us explain it in detail. In our models, we have two Higgs doublets H and H', which give 
the masses to the first two families and the third family of the SM fermions, respectively. 
Thus, H' T will have mass around a few hundred GeV, and it has interesting decay channels 
via Yukawa couplings, which will be discussed in the following. 

Second, without the non-renormalizable utd term, we have global symmetry U(5) x 
SU(3)' C x SU(2)' L x U(l)' Y in the above potential, and then we will have a PNGB during 
the SU(5) x SU(3)' C x SU(2)' L x U{l)' Y gauge symmetry breaking. To break the £7(5) global 
symmetry down to SU(5) and then give mass to 0, we do need this non-renormalizable 
term. Moreover, M* can be around the intermediate scale, for example, 1000 TeV. If we 
assume that all the high-dimensional operators are suppressed by the reduced Planck scale, 
i.e., M* = Mpi, we can generate the i/td term by introducing the XU field. The relevant 
Lagrangian is 

- C = (y T U^XU + y D n'U 2 D XU ] + H.C.) + M 2 XU \XU\ 2 , (2.2) 
where the mass scales fx' and M X u will be assumed to be around 1000 TeV. After we integrate 
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out XU, we get the needed high-dimensional operator 

v D -VJlBULulUl . (2.3) 
m xu 

We choose the following VEVs for the fields Ut and Ud 

< Ut >= v T | 13X3 I , < U D >= v D I ° 3X2 | , (2.4) 

02x3 I V ^2x2 

where Jj X j is the jxi identity matrix, and 0j X j is the ix j matrix where all the entries are 
zero. We assume that vd and Vt are in the TeV range so that the massive gauge bosons 
have TeV scale masses. 

From the kinetic terms for the fields Ut and Ud , we obtain the mass terms for the gauge 
fields 

(SPJjtfirUi) = \v T (gsAf - g' 3 Af) 2 + \vl (g,Af - g^Aff 



i=T,D 



V T , V L)\ ( Y 7al I Tal X 



+\& (4 + v 2 D ) (X,X, + Y,Y,) , (2.5) 
where g\ = v / 3fi'5/v / 5, and we define the complex fields (X M , Y^) with quantum numbers 



(3, 2, 5/6) from the gauge fields A" similar to that in the usual SU(5) model [12 ]. 

From the original gauge fields A™ and A™ and from Eq. ( 12. 5p . we obtain the massless 
gauge bosons A™ and the TeV scale massive gauge bosons (i = 3, 2, 1) which are in the 
adjoint representations of the SM gauge symmetry 



cos 9i sin 0i 
— sin 9i cos 9i 





( A™\ 







(2.6) 



where i — 3, 2, 1, and 



y 

sin% = ^ 5 , sinfl! = ^ 5 , (2.7) 



where j = 3,2. We also have the massive gauge bosons (X^, V M ) and (X M , V M ) which are 
not in the adjoint representations of the SM gauge symmetry. So, the SU(5) x S77(3)^ x 
SU(2)' L x t/(l) y gauge symmetry is broken down to the diagonal SM gauge symmetry 
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SU(3)c x SU(2) L x U(1)y, and the theory is unitary and renormalizable. The SM gauge 
couplings gj (j = 3,2) and g Y are given by 

111111 

J) ~ 7 5 + W & " Wf + W ( } 

If the theory is perturbative, the upper and lower bounds on the gauge couplings g 5 , g' 3 , 
g' 2 and g' Y are 

g 3 < g 5 < V4^ , g 3 < g' 3 < v 7 !^ , (2.9) 

. i . 9392 , n , nA 

92 < 9 2 < i 2 I , (2-10) 

V 91 ~ 92 

9V < , Jy < -^?m= . (2.1D 

V 3 9i - $g Y 

Note that the gauge coupling g 5 for SU (5) is naturally large at the TeV scale because the 
beta function of SU(5) is negative, i.e., SU(5) is asymptotically free. 



A. The Minimal Model 



We consider the minimal model first, where we do not introduce any extra ("X") particles 
XT, XU, Xf, ~Xf, XD, ID, XL, and XL. So the Higgs triplet H' T will be a few hundred 
GeV. We introduce the non-renormalizable operators to generate the quark CKM mixings. 
We also escape the proton decay problem by fine-tuning some coefficients of the higher- 
dimensional operators. 

The renormalizable SM fermion Yukawa couplings are 

-C = y^Q 3 H + y^N c k L J H + yf J D'rQ,H + y^L J H 

+y u 33 F 3 F 3 & + y£Fj 3 $ + yl 3 Nff z & + m^N? + H.C. , (2.12) 

where i/j = 1, 2, k/l = 1, 2, 3, and H = ia 2 H^ with a 2 the second Pauli matrix. Because 
the three right-handed neutrinos can mix among themselves via the Majorana masses, we 
can generate the observed neutrino masses and mixings. In addition, we make a wrong 
prediction that the bottom Yukawa coupling is equal to the tau Yukawa coupling at the 
low energy. We can easily avoid this problem by introducing the high-dimensional Higgs 
field under SU(5), which is out of the scope of this paper. In addition, the Yukawa terms 
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between the triplet Higgs field H' T in $ and the third family of the SM fermions are yf 3 t c b c H' Tl 
y 33 Q 3 L 3 H^, and y 33 t c r c H^,. So, we have (B + L) violating interactions as well. 

To generate the quark CKM mixings, we consider the higher- dimensional operators. The 
dimension-five operators are 

" C = W. {y* D * F ^ U T + V^sHUo + y d 3 J 3 Q t HU T + y^L^l/l) + H.C. (2.13) 
And the dimension-six operators are 

+4QWb)+H.C. . (2.14) 

Interestingly, if we neglect the dimension-six operators in Eq. ( 12.141) . we will generate the 
down-type quark mixings and charged lepton mixings via the dimension-five operators in 
Eq. ( 12.131) . Thus, the quark CKM mixing matrix can be realized via the down- type quark 
mixings. The proton decay is not a problem since there is no mixing between the top quark 
and up quark. For example, if we assume that the Yukawa couplings yf 3 and y 3i are order 
one and the VEVs of Ut and Ur> are about 1 TeV, we get M* ~ 1000 TeV to generate the 
correct CKM mixings. 

However, if we introduce the above dimension-six operators in Eq. (j2.14j) . proton decay 
can indeed arises due to the up-type quark mixings. For simplicity, we assume that yf 3 and 
y 3i are order one, M* ~ 1000 TeV, and the other Yukawa couplings y? 3 , y 3i , y™ 3 , y 3i are very 
small and of the the same order. Noting that the dimension-six proton decay operators have 
two up quarks, one down quark and one lepton, from the current proton decay constraints, 
we obtain that the Yukawa couplings yf 3 , y 3i , yf 3 , y 3i are about 10~ 4 . Because m e /m t ~ 10~ 5 , 
our fine-tuning is one order smaller and therefore is still acceptable. We would like to point 
out that the tau lepton decays to electron and muon will be highly suppressed due to the 
very small yf 3 and y 3i in the minimal model. 

B. The Renormalizable Model 

In the renormalizable model, we assume that all the non-renormalizable operators are 
suppressed by the reduced Planck scale. Thus, we need to introduce all the particles in 
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Table [H However, there are two exceptions: (1) We do not have to introduce the XT field 
since the triplet Higgs field H' T can have mass around a few hundred GeV; (2) We do not 
have to introduce the vector-like particles (XL, XL) since the neutrino masses and mixings 
can arise from the right-handed neutrino Majorana mass mixings. Then both the tau lepton 
decays to electron/muon and the proton decays to 7r°e + will be highly suppressed. 

The relevant renormalizable operators for the SM fermions are 

-C = F 3 Xj<$> + NpCf& + XDQiH + EfXLH + XfXDU T 

+J 3 XDU T + XDXfU T + D°XfU T + XLXfU D + xTJ 3 u D 
+XfXLU D + XfLiUl + VxfsTsXf + fixDiD^XD + fi XL J(LL t 
+M Xf XjXf + MxdXDXD + MxlXLXL + H.C. , (2.15) 

where we neglect the Yukawa couplings for simplicity. We assume that the mass terms 
M X f, M X d, and M X l are around 1000 TeV, while the mass terms fixj3, Hxm, and \ixu 
are relatively small. This can be realized via rotations of the fields since Xf, XD and XL 
only couple to one linear combinations of Xf/f 3 , XD/D^, XL/L i: respectively. Because 
the VEVs of Ut and Ud are around 1 TeV, the mixing terms from f 3 XDU Tl DfXfU},, 
XLf 3 UD, and XfLiUjj are small and negligible. 

For the dimension-five operators in Eq. (12. 13p . the yf 3 term can be generated from the 
above renormalizable operators D\X f\j\ and F 3 X /$, the y e i3 term can be generated from 
the above renormalizable operators E^XLH and XLf 3 UD, the y 3i term can be generated 
from the above renormalizable operators j ' 3 XDUt and XDQiH, and the y 3i term can be 
generated from the above renormalizable operators F 3 Xf$> and XfLillj^. 

In addition, we can show that there are no up-type quark mixings after we integrate out 
the vector-like particles. Let us explain the point. The SU(3)' C x SU{2)' L x U{l)' Y gauge 
symmetry can be formally embedded into a global SU(5)' symmetry. Under SU (5) x SU(5)', 
the bifundamental fields Ut and Ud form (5, 5) representation, the vector-like particles Xf 
and Xf respectively form (5, 1) and (5, 1) representations, and the vector-like particles 
(XD, XL) and (XD, XL) respectively form (1,5) and (1,5) representations. Because all 
these fields are in the fundamental and/or anti-fundamental representations of SU(5) and/or 
SU(5)', we cannot create the Yukawa interactions 10/10j5# or 10/10^5^' for the up-type 
quarks after we integrate out the vector-like particles. Therefore, there is no proton decay 
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problem. 

III. PHENOMENOLOGY AND SIGNALS AT LHC 

In this section we discuss the production mechanism for the exotic gauge bosons in our 
model and focus on the and vector bosons predicted in our model. These vector 
bosons carry both color and electroweak quantum numbers and behave as leptoquarks as 
well as diquarks. As the gauge bosons have their origins in the gauge group SU(5) which 
unifies only the third generation, as far as its coupling to fermions is concerned, it couples 
only to the third generation quarks and leptons. However, it interacts with the gluon as well 
as to all the other electroweak gauge bosons of the SM which would help in producing these 
particles at collider experiments. As far as their production at hadron colliders is concerned 
the dominant contributions would come from the strongly interacting subprocesses and 
therefore one can neglect the sub-dominant contributions coming from electroweak gauge 
boson exchanges. Note that they will however be produced only through the exchange of 
electroweak gauge bosons at electron positron colliders such as the International Linear 
Collider (ILC) [19] or the CLIC [3], envisioned and proposed for the future. We restrict 
ourselves to the study of these gauge boson at the currently operational LHC at CERN and 
therefore only focus on the couplings of the X M and vector bosons with the gluons which 
would be relevant for its production at the LHC. The general form of the interaction can be 
derived from the Lagrangian given by [21] 

£ = -\v%vr + M%v?vt - igsV^vJg^ (3.1) 

where V = X, Y and T a are the SU(3) C generators. The field strength tensors for the exotic 
vector fields and gluon are 

ST = d,G a u - d v G« + g s f ahc G, b G uc (3.2) 
Vf" = D*V vk - Dl% k (3.3) 

and the covariant derivative is defined as 

£>J" = - igsTjGl. (3.4) 
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FIG. 1: The tree level Feynman diagrams which contribute to the pair production of the and 
gauge bosons at the LHC, where both of them are denoted as V^. The subprocesses that contribute 
are (a) qq -> VV and (b) GG -»• VV. 

Using the above Lagrangian we derive the Feynman rules for the interactions of the lepto- 
quark gauge bosons V = X, Y with the gluon fields. These interactions then lead to the 
tree level Feynman diagrams as shown in Fig. ([I]) which contribute to the pair production of 
these exotic particles at the LHC. 



Calculation of cross sections 



Using Feynman rules for the interaction vertices of the exotic gauge bosons with gluons 
derived from Eq. fj3.1j) we can write down the full spin and color averaged matrix amplitude 
square for the quark-antiquark annihilation subprocess qq —> VV, (where q = u,d, c, s, b and 
V = X,Y) as 

^4 



\M 



9, 



9M*s 2 



[-12M V - s 2 t(s + t)+ 4M®(s + Qt) + 2M 2 s (2s 2 + 3st + 2t 2 ) 



—My (17 s 2 + 20st + 12t 2 )] 



while for the gluon induced subprocess GG — > VV, it is given by 
9M£ + 4s 2 + 9st + 9t 2 - 9M£(s + 2t) 



\M\ 



GG 



9s 



24s 2 (t - M 2 ) 2 (s + t-M, 



V 2 



[3M^ + 2s 4 - \2Myt + AsH 



+7s 2 t 2 + 6st 3 + 3t 4 + My (7s 2 + 6st + 18t 2 ) - 4M V (s 3 + 2sH + 3st 2 + 3t 3 )] . 



Note that the Mandelstam variables s and t are defined in the parton frame of reference. 
The pair production cross section at the parton level is then easily obtained using the above 
expressions. To obtain the production cross section we convolute the parton level cross 
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E CM =14TeV 
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FIG. 2: The production cross sections for pp — > VV at the LHC as a function of leptoquark mass 
My at center- of -mass energies, Eqm = 7,8 and 14 TeV. We have chosen the scale as Q = My, 
the mass of the leptoquark. 



sections a(qiqi — > VV) and a{GG — > VV) with the parton distribution functions (PDF). 

(3.5) 



a(pp ->• VV) = f dx ± j dx ? ^(^i-Q 2 ) x ^(^2,Q 2 ) x 6-(q iqi ->- VV) 



+ J dxi j dx 2 J* g (xi,Q 2 ) x J r g (x 2 ,Q 2 ) x a(GG -)■ VV), 

where J- q ., J z q i and J- g represent the respective PDF's for partons (quark, antiquark and 
gluons) in the colliding protons, while Q is the factorization scale. In Fig. (|2j) we plot the 
leading-order production cross section for the process pp — > VV at center of mass energies of 
7, 8 and 14 TeV as a function of the leptoquark mass My. We set the factorization scale Q 
equal to My, and have used the CTEQ6L1 PDF 22| . As seen from the plot, we find that the 
pair production cross section for both the X and Y leptoquark gauge bosons are quite big 
for significantly large values of their mass even at the 7 and 8 TeV runs of LHC. Thus one 
expects severe bounds on such particle masses from experimental data. In an earlier work 
18| , we had studied specific signals from the pair production of at LHC and put expected 
limits on its mass. This work was also followed up by the CMS experimental group which 
placed comparable limits on such leptoquark vector bosons {23 1 using collision data from the 
7 TeV run of the LHC. We note that as both the X and Y leptoquark gauge bosons have 
identical masses, any limits on one of them invariably leads to a similar limit on the other. 
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Thus it is important to explore all possible signals that come from the pair productions of 
these particles. In this work we extend our earlier study by looking at the different signals 
from the pair productions of such particles at LHC with center-of-mass energies of 8 TeV 
and 14 TeV. We note that at the 14 TeV run of LHC the production cross section for the 
leptoquark gauge bosons is significantly enhanced and would therefore improve the reach 
for such particle searches. 



B. Calculation of decays of the and gauge bosons 

To study the possible signals for the leptoquark gauge bosons, we need to know their decay 
properties. Since the third family of fermions is only charged under the gauge group SU(5), 
these leptoquark gauge bosons which come from the SU(5) gauge fields are only coupled 
to the third generation fermion fields. The interaction Lagrangian of the leptoquark gauge 
bosons and V M with the third generation fermions is given by 2^ |. 

+^=Y« [ - b Ral ^ c R - lurfrt + gfy-fbw] + B.C. (3.6) 

Using the above interaction Lagrangian, we can calculate the explicit decay modes of the 
leptoquark gauge bosons, where X^ decays to a top quark pair (tt) or anti-bottom quark + 
positively charged tau lepton (br + ) while V M has three decay modes to anti-bottom quark + 
a tau-neutrino (bu T ), anti-top quark + positively charged tau [ir + ) or top quark + bottom 
quark (tb). The partial decay width for each mode calculated using Eq.f l3.6p is then given 
by 

^«>-4£('-3)('-§r 

T(X -> br + ) = 9 i^- (3.7) 



T(Y ->■ ir + ) 
T(Y -> bv T ) 
r(Y -> tb ) 



gjMy 
24vr 

gjM Y 

12vr 
9\My 

24tt 



m t 



My" 



2 + 



2 + 



m t 



(3. 
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where g 5 is the 577(5) gauge coupling and we have only kept the top quark mass (m t ) and 
neglected the other fermion masses. We plot the branching fractions of the leptoquark gauge 
bosons decays as well as their total widths, as shown in Fig.(|3]). It is interesting to note 
that while the X M decays dominantly to br + , it also has a substantial branching fraction to 
a pair of same sign top quarks. For very large values of the mass Mx of the X^, when the 
mass of the top quark can be neglected, we find that ^fe^tT — ^ ^ or ^ ne ^ leptoquark 
gauge boson we find that for smaller values of its mass it has the dominant decay fraction to 
bv T while its decay to tr + and tb are equal. But for My quite large such that the top quark 
mass may be neglected, all decay modes have the same branching probability of 1/3. 
With the knowledge of the decay modes of the leptoquark gauge bosons and the branching 
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FIG. 3: Illustrating the decay branching fractions of the leptoquark gauge bosons (a) X„, (b) Y^ 
and, (c) the total widths as a function of their mass. 



fractions for the decays we can now analyze all the different final states that we expect from 
the pair production of these leptoquarks at the LHC Note that the total decay widths (ry) 
for the leptoquark gauge bosons are such that Ty < O.lMy and we have therefore used 
the narrow width approximation (NWA) which proves to be a useful tool in simplifying the 
analysis without introducing large errors 25]. We have fixed the g$ coupling to the value of 
the strong coupling constant (g s ) throughout the analysis. 



C. Signals at LHC 



In Ref.|l8| we studied the signals for the pair production of the leptoquark gauge 
bosons and their subsequent decays into the dominant mode br + at LHC center of mass 
energies of 7 TeV and 8 TeV. The final state signal was bbr + r~ with all the four particles 
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being detected in the respective flavor tagged mode. It was observed that the signal stands 
out as resonances in the invariant mass distribution of the r lepton paired with the b jets 
against the continuum SM background, provided all the four final state particles carried 
significant transverse momenta. Using this signal a phenomenological prediction on the 
LHC reach was made on the mass of the which has subsequently been estimated as 



760 GeV at 95 % C.L. by the CMS Collaboration [23j at LHC with 7 TeV center of mass 
energy. As our model predicts another decay mode (to top quark pairs) for the gauge 
boson, where the X^ behaves as a diquark, carrying quantum numbers of two quarks, it is 
of extreme importance to be able to highlight this characteristic which distinguishes this 
particle from the usual leptoquark particles. Establishing the existence of both decay modes 
is needed to show that these interactions are both baryon and lepton number violating. It 
is also worth pointing out that a similarly massive Y" M in the spectrum which couples as 
strongly to the gluons as the X^ will also be produced with similar rates and needs to be 
studied in tandem with the production of the X^ particles at the LHC. 

We now consider all decay modes of both the X^ and and discuss final states which 
is then studied against the SM backgrounds. For the pair production process of X^ gauge 
bosons, where X — > br + ,tt we have the following different final states given as 

pp — > XX — > br + br-, ttbr', br + tt, tttt. (3.9) 

The top quark would further decay, either semileptonically or hadronically to give multi- 
lepton and high jet multiplicity final states. For our purposes, if we assume that the top 
quarks could be reconstructed with some reasonable efficiency in either modes, we can just 
focus on the above mentioned final state signal. Similarly for the pair production of the Y^ 
gauge bosons, where Y — > bv T , ir + , tb we get the following set of final states given as 

pp — > YY — > bb$ T) ltr~$ T , ibr + $ T , bbi$ T , bbt$ T , 

^ tir + T-, iibr + , ttbr~, tibb. (3.10) 

Note that both the X^ and Y^ gauge boson productions at the LHC leads to a rich range 
of diverse final states which lead to many multi-particle signals and would lead to distinct 
resonances in the invariant mass distributions in some pairs corresponding to the mass of the 
X^ and Y^ states. Notably we find that each particular event rate is fixed once the model 
parameters have been fixed, which in our case is the mass of the leptoquark gauge bosons 
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while its coupling strength to the gluons has been fixed to be the strong coupling constant. 
Thus the success of the model is not dependent on an observation in only one particular 
final state but that observation needs to be complemented simultaneously in various other 
channels as listed above in Eqs.f l3.9j) and (13.10!) . Thus the study on all simultaneous channels 
deserves merit as it will be able to confirm or falsify the model in question. 

We now consider different final states and analyze the signals against the SM background. 
As we expect that the new gauge bosons when produced on-shell will decay to specific final 
state products, this would lead to a bump in the invariant mass distribution of the decay 
products. Keeping this in mind, it is instructive to first consider the most likely signals where 
the resonances would be observable. Based on the decay channels and final states listed in 
Eqs. (I3.9j) (13. lOj) one should consider the (6r) mode for the gauge bosons while the (tb and 
tr) mode looks the more promising for the resonance searches. The other modes either 
involve neutrinos or more than a single top quark in the final state, which further decays 
either semileptonically or hadronically and therefore makes it more tasking to reconstruct the 
leptoquark gauge boson mass. However, we must emphasize that for measuring the electric 
charge of these gauge bosons one definitely requires that the X M resonance is observed in 
the invariant mass distribution of same-sign top quark pair (tt) while the resonance is 
observed in the (tr~) final state or its charge conjugate mode. Notwithstanding the fact that 
reconstructing the tt state would be challenging, it would definitely lead to a very interesting 
observation. Final states involving b jets require measuring the b jet charge which looks to 
be more difficult and hence not a desired mode to get information on the charge of the exotic 
gauge bosons. 



Signal 


SM 


Signal 


SM 


2br+T- 


2br + r~; 2jr + T~; bjr + T~ 


ttbr~, ttbr + 




ttu 


tttt 


tir + T~ 


ttT + T~ 


2bti 


2btt; 2jti; bjti 


2b$ T 


2b$ T - 2j$ T - jb$ T 


btr-$ T 


Ut~$ t ; jtT~$ T 


bir + $ T 


btT + $ T ] jtT + $ T 


2bt$ T 


2jt$ T ; bjt$ T 


2bt$ T 


2ji$ T - bjt$ T 



TABLE II: Illustrating the final state signals and the corresponding SM background subprocesses. 
Note that $ T for the SM subprocesses represents one or more neutrinos in the final state. 
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In Table HT1 we list the relevant SM background subprocesses that we have considered for 
each set of final states for the signal. Note that we do not make a distinction between the 
b and b but we distinguish between a r + and r~ by assuming exact charge measurement 
will be possible. We also distinguish between a top quark and anti-top quark assuming that 
they will be reconstructed with their respective charge identifications from its semileptonic 
decay modes. We associate an efficiency factor of e t with this reconstruction. For final state 
signals not involving neutrinos we have not considered SM subprocesses with $ T as they will 
involve extra electroweak vertices which suppress the contributions and further requirements 
on missing transverse momenta would make these contributions too small to take into further 
consideration. We highlight the above mentioned invariant mass distributions in our model 
for a few choices of the X M and gauge boson masses considered at two different center 
of mass energies for the LHC We focus our attention to the recently concluded 8 TeV run 
and the proposed upgrade in energy of 14 TeV for the LHC. As a current limit of 760 GeV 
exists on the leptoquark gauge boson mass from the CMS analysis [23( we choose a mass 
of 800 GeV to show the distributions at the 8 TeV run of LHC while a larger mass of 1 
TeV is chosen to highlight the signal distributions at the 14 TeV run. We note that there 
are more than one set of final states where a particular resonance could be observed in the 
invariant mass distributions and so we consider the scenario where we look at a few definite 
invariant mass distributions in individual final state modes listed in Eqs. fl3.9p and (13.101) . 
We list below the pair of final state particles for which the invariant mass distribution is 
considered, motivated by favored modes for reconstructing the mass and the charge of the 
X^ and gauge bosons. 

(CI) Invariant mass distribution of br~ coming from the final states br + br~, ttbr~ . This 
is the most favorable mode for reconstructing the resonance. 

(C2) Invariant mass distribution of same sign top quark pair tt coming from the final states 
ttbr~, tttt. The reconstruction of the leptoquark mass in this mode, although difficult, 
is essential in measuring the charge of the X^. 

(C3) Invariant mass distribution of tb coming from the final states 
tibb, ttbr', tbT~$ T , bbt$ T . This is one of the favorable modes for reconstructing the 
Y^ resonance. 
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(C4) Invariant mass distribution of tr~ coming from the final states tir~T + , ttbr~, btr~$ T . 
This mode is essential to measure the charge of the Y^. Note that the tr~ resonance 
corresponds to the charge conjugate mode of Y^. 

We shall now discuss the signal and the associated SM backgrounds for the list of res- 
onances given by CI— C4. Note that the signal subprocesses which contribute to give a 
br~ final state as listed in (CI) come from both and Y^ pair productions. However the 
resonant distribution only happens for the production while the Y^ contribution acts 
to smear out the resonance although it does contribute in enhancing the signal over the 
SM background. A further smearing effect would come if the tbr~$ T signal is included. 
But we can reject that contribution by demanding that we don't include events with large 
missing transverse momenta in the final state when reconstructing the br~ invariant mass. 
As discussed in Ref.[18| the dominant background for the resonant signal in the br channel 
comes from pp — > 2b2r, 46, 2j2b, 2j2r, 4j, ti where j = u, d, s, c when we consider the signal 
coming from the pair production of X^ which then decay in the br mode to give a 2br + r~ 
final state. The light jet final states in the SM can be mistaged as r or b jets and thus form 
a significant source for the background due to the large cross sections at LHC, as they are 
dominantly produced through strong interactions. Guided by previous analysis [18], we note 
that a very strong requirement on the transverse momenta for the b jet and the r lepton is 
very helpful in suppressing the SM background. The SM background has been estimated 
using Madgraph 5 [26]. In this analysis we further restrict the number of SM background 
sub-processes that contribute to the final state with br~ by demanding that the tau charge 
is measured. Therefore we neglect the contributions coming from jets that fake a tau. For 
example, when we consider the final state as 25r + r~ and demand that the tau lepton is 
tagged as well as its charge measured, we include pp — > 2b2r, 2j2r, ti as the dominant SM 
processes for the background. 

We have used two values for the leptoquark gauge boson (V = X, Y) masses, My = 800 
GeV at LHC with center of mass energy 8 TeV and My = 1 TeV at LHC with center of 
mass energy 14 TeV to highlight the signal cross sections and differential distributions for 
invariant mass. We set the factorization and renormalization scale (Q = Mz) to the mass 
of the Z boson and also use the strong coupling constant value of a s evaluated at the Z 
boson mass. Note that we have evaluated the individual signals as listed in Eqs.f l3.9p and 
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Variable 


Cut d at 8 TeV 


Cut C 2 at 14 TeV 


P T T J 


> 80 GeV 


> 200 GeV 


$T 


> 100 GeV 


> 200 GeV 


M 


< 2.5 


< 2.5 




> 0.4 


> 0.4 


M jj,T+T- 


> 5 GeV 


> 5 GeV 



TABLE III: Two different set of cuts, d at LHC with y/s = 8 TeV and C 2 at LHC with x/s = 14 
TeV, imposed on the final states listed in Eqs. ^3. 9\) - $3.10\) where the cuts on p T applies only to 
final states with neutrinos in the decay chain. 

(I3.10p against their specific backgrounds independently. We have assumed in our analysis 
that the top quark and the anti-top quark are reconstructed with good efficiencies which 
we can parameterize as e t - Note that we have used the following efficiencies for b and r 
tagging, e& = e T = 0.5 while we assume a mistag rate for light jets to be tagged as b jets as 
1% and c jets tagged as b jets to be 10%. All our results here are done at the parton level 
and therefore to account for the detector resolutions for energy measurement of particles, 
we have used a Gaussian smearing of the jet and r energies with an energy resolution given 
by AE/E = 0.8/^/E (GeV) and AE/E = 0.15/y/E (GeV) respectively when analyzing 
the signal events. 

In Table II I II we list the kinematic selection cuts on the events. As the primary decay 
modes of the heavy leptoquark gauge bosons will have very large transverse momenta we 
put strong cuts on them. This helps in suppressing the SM background while it does not 
have any significant effect on the signal events. The cuts on $ T applies only to final states 
with neutrinos in the decay chain while the ARij cut is on any pair of visible particles. The 
invariant mass cut Mjj is on any pair of jets in the final state. 

With the above set of kinematic selection on the final state events we evaluate the signal 
cross sections and the corresponding SM background given in Table [III We first consider the 
resonance given by (CI) and show the invariant mass distribution of br~ in Fig. HI We 
must point out here that the r~ is paired with the b jet which has the leading transverse 
momenta in case there exist more than one tagged b jets. After including the efficiency 
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factors eb and e T associated with tagging the b and r jets and mistag rates, we estimate 
the signal cross section in the 26r + r~ mode as 4.23 fb for Mx,y — 800 GeV at LHC with 
y/s = 8 TeV and 12.05 fb for M x ,y = 1 TeV at LHC with y/s = U TeV. In Figfjflwe plot the 
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FIG. 4: Invariant mass distribution of br~ for the signal and SM background for two different 
choices of leptoquark gauge boson mass, (a) My = 800 GeV considered at LHC with y/s = 8 TeV 
and (b) My = 1 TeV considered at LHC with y/s = 14 TeV. 

invariant mass distribution for the signal. The dominant SM background are given by the 
following subprocesses, cr(2tVr + T~) ~ 1.8 fb, <j(2ct + t~) ~ 1.6 fb and a(2jr + T~) ~ 167.6 fb 
which after including the efficiency factors, mistag rates is added to give 0.119 fb. This is 
plotted in Fig.flU) as "SM (2tVr + r - )". The corresponding SM background at 14 TeV center 
of mass energy is much more suppressed (~ 0.002 fb) because of the strong requirement on 
the transverse momenta of the jets and the charged tau leptons. The signal is clearly seen to 
stand out as resonance and one therefore expects this particular mode to be very favorable 
in searching for the resonance by suppressing the SM background by demanding r lepton 
charge identification which gets rid of the large all jet background. Another mode for the 
br~ resonance which has completely negligible SM background, is for the final state ttbr~. 
There are two different sources for the signal in this case, one which corresponds to the final 
states coming from the XX pair production while the other from the YY pair production. 
As the YY contribution does not lead to a resonance in the br~ mode, it will act to smear 
out the resonance as compared to that seen for the 2br + r~ final state. This is evident in 
Fig. (HJ) where the width of the resonance is seen to spread out in more invariant mass bins 
for the ttbr~ final state. Assuming a top reconstruction with an efficiency of e t we find that 
the signal cross section from XX for M x = 800 (1000) GeV at LHC with y/s = 8 (14) 
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TeV is 8.19 (28.23) x e\ fb while the signal cross section from YY for M Y = 800 (1000) 
GeV at LHC with y/s = 8 (14) TeV is 4.04 (13.8) x e\ fb. Note that the r and b tagging 
efficiencies have been already included. In Fig. (HJ) we have assumed St — 1 for illustration 
purposes. Therefore the efficacy of the signal with the same sign top pairs in the final state 
is dependent on the inherent purity of the top reconstruction at experiments. 

We now consider the resonance given by (C2) and show the invariant mass distribution 
of the same sign top pair tt in Fig. ([5]). As pointed out earlier, this mode is necessary to 
measure the charge of the leptoquark gauge boson mass. A resonant bump in the same 
sign top pair invariant mass distribution would be a clear indication of a particle decaying 
into two same sign top quarks and therefore give a strong indication that the particle carries 
4/3 electric charge and has quantum numbers of a diquark. The signal is again considered 
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FIG. 5: Invariant mass distribution of same sign top pair tt for the signal and SM background for 
two different choices of leptoquark gauge boson mass, (a) My = 800 GeV considered at LHC with 
^ = 8 TeV and (b) My = 1 TeV considered at LHC with y/s = 14 TeV. 



for two different set of final states, both of which show an invariant mass peak in the same 
sign top quark pair. In the tttt final state the signal cross section comes solely from the pair 
production of the XX gauge bosons. As we have assumed a reconstruction efficiency for the 
top quarks as e t , the cross section for M x = 800 (1000) GeV at LHC with y/s = 8 (14) TeV 
is 14.93 (62.81) x e\ fb. The SM background for the same subprocess is 2.31 (24.34) x ej fb 
at LHC with y/s = 8 (14) TeV. Although the strength of the signal crucially depends on the 
reconstruction efficiency, even a low efficiency in the long run will lead to a very important 
observation provided similar resonances are observed in the br~ or br + final states. The other 
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final state which shows a bump in tt invariant mass is ttbr~ and its strength was already 
discussed for Fig. (J3J). Note that again the YY contribution does not help the resonance, but 
is effective in enhancing the signal in this mode. 
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FIG. 6: Invariant mass distribution oftb for the signal and SM background for two different choices 
of leptoquark gauge boson mass, (a) My = 800 GeV considered at LHC with yfs = 8 TeV and (b) 
My = 1 TeV considered at LHC with ^/s = 14 TeV. 



We now look at the final states which correspond to resonant signals for the Y gauge 
boson. We therefore consider the resonance given by (C3) and show the invariant mass 
distribution of the top-bottom pair tb in Fig. ([6]). Note that one of the dominant decay 
mode for the Y^ gauge boson gives neutrinos in the final states that leads to large missing 
transverse energy (MET) and is not suitable to reconstruct the Y^ mass. However, allowing 
one Y to decay in the neutrino mode still allows reconstruction of the other in the visible 
decay modes of tb and tr. A large MET in the final state also helps in suppressing large 
contributions to the SM background through all hadronic final states which proceed through 
strong interactions. In Fig. ((6]) we consider four different final state signals which lead to a 
resonance in the tb invariant mass, namely ttbb, ttbr~ , tbr~$ T and tbb$ T . The signal ttbr~ 
remains the same as discussed for Fig.flU) with the only difference being that the contribution 
coming from the XX pair production now acts to smear out the resonance in tb invariant 
mass distribution coming from the Y^. This is the cleanest mode with practically no SM 
background, although depending on the reconstruction of the top quarks. The signal cross 
section for the ttbb final state comes from the Y^ pair production and for My = 800 (1000) 
GeV at LHC with y/H = 8 (14) TeV is 4.04 (13.75) x e\ fb. The SM background at 
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LHC with y/s = 8 (14) TeV for the signal comes dominantly from three subprocesses with 
a(tibb) ~ 54.4 (34.1) fb, a(ticc) ~ 55.1 (34.4) fb and ff(ttjj) ~ 10.14 (7.45) pb. The 
stronger cuts at the 14 TeV run is responsible for the relatively smaller numbers for the SM 
background for the higher energy run. Note that after including the tagging efficiencies and 
misstag rates, the corresponding SM background for the tibb final state comes out to be 
15.18 (9.65) x e\ fb. Although the SM backgrounds are large in this case, the differential 
cross section is seen to fall rapidly for larger values of the invariant mass. Therefore, a 
strong cut on the tb invariant mass will be useful to suppress the background further. For 
the two final states involving missing transverse energy, we have combined their contribution 
in Fig. © under the signal ll tbr~(b) + MET 1 . We find that the SM background for tbb$ T 
is completely negligible. Note that in the SM background for u tbr~(b) + MET" , Fig. ([6b ) 
shows an unusual kink in the invariant mass distribution of This is a kinematical effect 
driven by the strong kinematic cuts that we put on the final states. The contribution to final 
state events for tbr~ + MET can be isolated into dominant contributions coming from the 
on-shell production of tt and contributions where a 6-jet is recoiling against a tW~ system 
(2 — > 3 scattering). The strong requirement on the p? > 200 GeV of the final products 
suppresses the tt contributions more while affecting the btW~ less which leads to the kink in 
the invariant mass distribution. A much weaker requirement of pt > 80 GeV (as is the case 
with y/s = 8 TeV) or lower leads to complete amelioration of the kink like behavior and the 
SM background distribution looks very similar to that in Fig.((6k). The large contribution 
to the background comes from the a{tbT~$ T ) ~ 88.7 (3.05) fb at LHC with y/s = 8 (14) 
TeV, while the tcr~$ T and tjr~ $ T are much suppressed due to the small CKM mixings 
between the first two generation quarks and the top quark. The SM background after 
including the efficiency factors is then given as 22.16 (0.75) x e t fb, while the signal for 
My = 800 (1000) GeV at the two center of mass energies is a(tbb$ T ) = 4.21 (13.21) x e t fb 
and a{tbr~ $ T ) = 4.21 (13.20) x e t fb. Note that tbb$ T is the one which gives a resonant 
signal while tbr~ $ T gives a continuum in the tb invariant mass distribution because the t and 
b come from different (Y — > bu T , Y — > tr~). This can be seen in Fig. ([6]) where the large 
signal contribution in the tbr~$ T channel is spread out in the invariant mass distribution. 
Therefore it is instructive to put a r veto on the signal with missing transverse momenta 
when looking at the invariant mass distribution in tb. Again for illustrative purposes we 
have chosen Ef — 1. 
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FIG. 7: Invariant mass distribution of tr~ for the signal and SM background for two different 
choices of leptoquark gauge boson mass, (a) My = 800 GeV considered at LHC with \fs = 8 TeV 
and (b) My = 1 TeV considered at LHC with \fs = 14 TeV. 

We finally consider the resonance given by (C4) which again is essential in measuring 
the charge of the Y M gauge boson. To measure the charge one requires the charge mea- 
surement of the r lepton as well as the reconstruction of the top quark in its semileptonic 
channel. We therefore show the invariant mass distribution in the reconstructed top quark 
and charged tau lepton pair (tr _ ) in Fig .(171) which corresponds to a resonance for the charge 
conjugated field of Y^. The signal is obtained from three different set of final states given 
by tir + T~ , tbr~$ T and Itbr^ . As discussed before the 2tbr~ contribution is found to have 
negligible SM background but the contribution from the XX production to the tr~ invariant 
mass distribution itself acts as a background for the resonant signal from the YY production. 
The tir + T~ signal comes solely from the Y^ pair production and we find that with the proper 
charge identification of the r leptons, we can ignore contributions from SM background pro- 
cesses such as ttjj. The signal cross section in this mode is found to be 4.04 (13.81) x e\ fb 
for My = 800 (1000) GeV at LHC with yfe = 8 (14) TeV. The SM background is quite 
suppressed at both center of mass energy values, given by 0.35 (0.04) x e\ fb. The tbr-$ T 
signal discussed for the tb resonance in Fig. ([6]) was found to give a continuum distribution in 
the tb invariant mass. However it leads to a resonance in the tr~ invariant mass distribution 
as Y — > tr~ . The event rates are the same as before but one can clearly see a distinct 
resonance confined to a few bins in the invariant mass distribution of tr~ in Fig. (J7J) for the 
tbr~ $ T signal. The large SM background for this mode can again be suppressed with a sig- 
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nificantly strong cut on the tr~ invariant mass. Note again that for the SM background for 
the M tT - invariant mass distribution at ^/s = 14 TeV, a similar kink like feature is observed 
in Fig. ([7b). This is because the same subprocess which contributes in Fig.flBb) also features 
in this case with similar kinematic cuts which we have already discussed before. We must 
however point out that if strong px requirements for the final products were put on events 
for a/s = 8 TeV, we get a similar kink like behavior in the invariant mass distribution. 



D. LHC sensitivity to the and gauge bosons 

As evident from our analyses of the resonant signals for the and gauge bosons 
in our models, the LHC would be able to see the signals in various different channels for 
significantly large values of their mass. A single channel analysis in the br mode relevant 



for search was considered for its search at the 7 TeV run of LHC |l8l . |23 | while another 
experimental study relevant for the Y v search in the bb$ T channel has been done by the 
CMS Collaboration j^. Here we do a more expansive sensitivity reach at the LHC for 
these gauge bosons that can be obtained at different integrated luminosities. For the top 
decaying semileptonically to M + U£ where £ = e, /i the events will be at most, or less than 
~ 22% of the reconstructed top events. While it would be ~ 66% in the hadronic decay 
mode. Thus it gives a clear demarcation on the event rate we specify for the final states 
involving the top and anti-top quarks that would lead to any signal events to reconstruct 
the tops. 

For the sensitivity analysis we define the signal to be observable if the lower limit on the 
signal plus background is larger than the corresponding upper limit on the background {28 1 
with statistical fluctuations 



L(a s + a b ) - N^L(a s + a b ) > La b + N^u7 h 

or equivalently, 



N 

a s >j- 



N + 2^/L^] } (3.11) 



where L is the integrated luminosity, a s is the signal cross section, and a b is the background 
cross section. The parameter N specifies the level or probability of discovery. We take 
N = 2.5, which corresponds to a 5cr signal. For a b ^> <j s , this requirement becomes similar 
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Final State 


°~SM (fb) 


Final State 


°~SM (fb) 


2&T+T- 


0.12 (0.002) 


ttbr', ttbr + 




tttt 


2.31 (24.34) 


tir + T- 


0.35 (0.04) 


2bti 


15.18 (9.65) 


2b$ T 


25.06 (3.83) 


btr~$ T 


22.16 (0.75) 


btT + $ T 


22.16 (0.75) 


2bt$ T 


0.003 (0.001) 


2bt$ T 


0.001 (0.0006) 



TABLE IV: The combined SM cross sections estimated at parton level using MadGraph 5 for the 
different final state signals at LHC with y/s = 8 TeV and ^fs = 14 TeV. The 14 TeV values are 
given in parenthesis. Note that the cross sections given satisfy the kinematic cuts listed in Table 
and all tagging efficiencies and misstag rates are included. 



to 

S = = —=J= > 5, 3.12 

where N s is the number of events for the signal, Nj, is the number of events for the back- 
ground, and S equals the statistical significance. 

In Table IIVI we have calculated the SM background for the different final states that we 
have considered for the signal coming form the pair productions of the and gauge 
bosons. The cross sections shown in Table [IV] are obtained after passing the events through 
the kinematic selection conditions given in Table IIHI In most cases the SM backgrounds 
are quite small and would remain negligible even with an integrated luminosity of 100 fb -1 . 
Note that as the top reconstruction would require sufficient events after it has decayed, we 
need much larger cross sections for the final states involving top quarks. To use Eq.f l3.lll) . 
we require the background events to be sufficiently large such that the fluctuations to a 
Gaussian distribution could be applied. We find that the best reaches are obtained for the 
bb$ Tl bir + $ T and btr~ $ T final states. For the bb$ T final state at LHC with y/s = 8 TeV, 
the signal cross section for a 5a sensitivity must be greater than 8.54, 5.91, 4.78 fb for an 
integrated luminosity of L = 10, 20, 30 fb^ 1 respectively. This corresponds to the mass 
reach of My = 737, 772, 793 GeV respectively. With the higher center of mass energy 
option for LHC with ^/s = 14 TeV, the signal cross section for a 5a sensitivity must be 
greater than 1.995, 1.041, 0.586 fb for an integrated luminosity of L = 30, 100, 300 fb^ 1 
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respectively. These lead to a mass reach of 1325, 1440, 1545 GeV respectively. For the other 
channels involving the top quark in the final state, we assume the reconstruction efficiency 
for the top quark e t — 0.5 which includes the event loss from kinematic cuts after the top 
decays. Adding the contributions for bir + $ T and btr" $ T we find that at the 8 TeV run 
of LHC, the mass reach is 770, 795 GeV for an integrated luminosity of L = 20, 30 fb~ l 
respectively while at the 14 TeV run of LHC, where we use the high luminosity options of 
200 fb- 1 and 300 /ft -1 , the 5a sensitivity comes out to be about 1650 GeV and 1690 GeV 
respectively. 



IV. SUMMARY AND CONCLUSIONS 



Although the Standard Model, based on local gauge symmetries, accidentally conserve 
baryon and lepton numbers, there is no fundamental reason for the baryon and lepton 
numbers to be exact symmetries of Nature. In fact, Grand Unification, unifying quarks and 
leptons, naturally violate baryon and lepton number. The remarkable stability of the proton 
dictate that the masses of these leptoquark and diquark gauge bosons to be at the 10 16 GeV 
scale. However, baryon and lepton number violating interaction involving only the 3rd family 
of fermions is not much constrained experimentally. Inspired by the topcolor, topflavor and 
top hypercharge models, we have a top-GUT model where only the third family of fermions 
are unified in an S77(5) with the symmetry breaking scale at the TeV. These models give 
baryon and lepton number violating gauge interactions which involve only the third family, 
and with interesting resonant signals at the LHC. 

We have proposed two models, the minimal and renormalizable top S77(5) where the 
S77(5) x SU(3)' C x SU(2)' L x U(1)' Y gauge symmetry is broken down to the Standard Model 
(SM) gauge symmetry via the bifundamental Higgs fields at low energy. The first two 
families of the SM fermions are charged under SU(3)' C x SU(2)' L x U(1)' Y while the third 
family is charged under S77(5). In the minimal top 577(5) model, we showed that the quark 
CKM mixing matrix can be generated via dimension-five operators, and the proton decay 
problem can be solved by fine-tuning the coefficients of the high-dimensional operators at the 
order of 10~ 4 . In the renormalizable top S77(5) model, we introduced additional vector-like 
fermions whose renormalizable interactions with the SM particles generate these dimension 5 
interactions and we can explain the quark CKM mixing matrix by introducing the vector-like 



29 



particles, and also there is no proton decay problem. We have discussed the phenomenology 
of the models in details looking for the resonant signals for the baryon and lepton number 
violating leptoquark as well as diquark gauge bosons at the LHC, as well as the various final 
state arising from the productions and decays of these heavy gauge bosons. We have also 
calculated the corresponding SM backgrounds. We find that a 5a signal can be observed 
for a mass leptoquark / diquark of about 770/800 GeV at the 8 TeV LHC with luminosity 
of 20/6" 1 /30/6^ 1 . The mass reach extends to about 1450 TeV for 14 TeV LHC with a 
luminosity of 100/6 -1 . 
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